A Numerical Study on Learning Curves in Stochastic
نویسندگان
چکیده
The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multi-layer feed-forward networks trained with back-propagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1=t as predicted. For a medium range t a faster 1=t 2 scaling is observed. This eect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from ineective to eective learning.
منابع مشابه
Elite Opposition-based Artificial Bee Colony Algorithm for Global Optimization
Numerous problems in engineering and science can be converted into optimization problems. Artificial bee colony (ABC) algorithm is a newly developed stochastic optimization algorithm and has been widely used in many areas. However, due to the stochastic characteristics of its solution search equation, the traditional ABC algorithm often suffers from poor exploitation. Aiming at this weakness o...
متن کاملAn integrated vendor–buyer model with stochastic demand, lot-size dependent lead-time and learning in production
In this article, an imperfect vendor–buyer inventory system with stochastic demand, process quality control and learning in production is investigated. It is assumed that there are learning in production and investment for process quality improvement at the vendor’s end, and lot-size dependent lead-time at the buyer’s end. The lead-time for the first batch and those for the rest of the batches ...
متن کاملStudy on efficiency of the Adomian decomposition method for stochastic differential equations
Many time-varying phenomena of various fields in science and engineering can be modeled as a stochastic differential equations, so investigation of conditions for existence of solution and obtain the analytical and numerical solutions of them are important. In this paper, the Adomian decomposition method for solution of the stochastic differential equations are improved. Uniqueness and converg...
متن کاملA numerical study on learning curves in stochastic multilayer feedforward networks.
The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t2 scaling is observed. This effect...
متن کاملNumerical Solution of Weakly Singular Ito-Volterra Integral Equations via Operational Matrix Method based on Euler Polynomials
Introduction Many problems which appear in different sciences such as physics, engineering, biology, applied mathematics and different branches can be modeled by using deterministic integral equations. Weakly singular integral equation is one of the principle type of integral equations which was introduced by Abel for the first time. These problems are often dependent on a noise source which a...
متن کامل